Reinforcement Learning with PI2 Algorithm to Generate Motor Primitives of a Complex Snake-Like Robot

In this thesis work a policy improvement algorithm called Policy Improvement with Path Integrals (PI2) is used to generate goal-directed locomotion of a complex snake-like robot with screw-drive units. PI2 is numerically simple and has an ability to deal with high dimensional systems. Here, PI2 is used to find proper locomotion control parameters, like joint angles and screw-drive unit velocities, of the robot. The learning process is achieved in simulation and the learned parameters are successfully transferred to the real robot. As a result the robot can locomote towards a given goal. Furthermore, as PI2 generates proper control parameter sets for different goals and robot body shapes (like, straight-line, zigzag, arc), a considerable and meaningful repertoire of robot behaviors is obtained in this way. Out of this, certain learned parameters sets are selected as motor primitives to generalize goal-directed locomotion generation in this work and generate new behaviors online. By selecting different primitives and properly chaining or combining them along with parameter interpolation, the robot can successfully handle complex tasks like, reaching a single goal or multiple goals while avoiding obstacles and compensating to a change in its body shape. As a result, the robot can successfully locomote towards a given goal as well as handle a complex environment or an unknown situation.